Skip to content

Sort verses in JSON data files at source#280

Open
samanyougarg wants to merge 4 commits intomainfrom
sorted-jsons
Open

Sort verses in JSON data files at source#280
samanyougarg wants to merge 4 commits intomainfrom
sorted-jsons

Conversation

@samanyougarg
Copy link
Member

Summary

  • Updated migration script to sort verses when generating v4 data
  • Regenerated all common (7 languages) and author (28 files) JSON files with properly sorted verses
  • Removed redundant sorting logic from web UI code
  • Updated Flutter app data with sorted JSON files and regenerated compressed assets

Why This Change?

Previously, verses were not sorted in the JSON files themselves. This meant:

  1. Every client (web, mobile) had to implement sorting logic
  2. Potential for inconsistencies if sorting logic differed
  3. Extra runtime overhead for sorting on every page load

Now verses are sorted once at build time in the data generation script, ensuring consistency across all platforms.

Changes Made

Data Generation (gita-data-2.0)

  • Added sorting logic to migrate_to_v4.py that sorts verses by their starting number
  • Handles both single verses (e.g., "7") and ranges (e.g., "4-6") correctly
  • Regenerated all v4 data files

Web App (bg-frontend)

  • Removed .sort() call from src/lib/data/queries.ts
  • Removed sorting helper and logic from verse page.tsx
  • Added comments noting that data is pre-sorted

Flutter App (gita-flutter-2.0)

  • Copied sorted JSON files to assets/data/
  • Regenerated all compressed .gz files in assets/compressed_data/

Testing

  • ✅ Verified verses are in correct order: 1, 2, 3, 4-6, 7, 8... (verse 7 now appears after 4-6, not at the end)
  • ✅ Tested navigation between verses works correctly
  • ✅ Both web and mobile apps now have consistent verse ordering

Previously, verses were sorted in the UI code after loading from JSON files. This meant every client (web and mobile) had to implement their own sorting logic.

Changes:
- Updated migration script to sort verses by their starting number when generating v4 data
- Regenerated all common and author JSON files with sorted verses
- Removed sorting logic from web UI (queries.ts and verse page.tsx)
- Updated Flutter app with sorted data and regenerated compressed files

Benefits:
- Single source of truth for verse ordering
- Consistent ordering across web and mobile apps
- Simplified client code by removing redundant sorting logic
- Better performance as sorting is done once at build time
@vercel
Copy link

vercel bot commented Dec 8, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Comments Updated (UTC)
bg-frontend Error Error Dec 9, 2025 7:35pm
gita-frontend-v2 Ready Ready Preview Comment Dec 9, 2025 7:35pm

Fixed inconsistent newline formatting in Sanskrit text that was causing:
1. Unwanted gaps between lines of the same verse (had \n\n, should be \n)
2. Verses running together in ranges (had \n, should be \n\n between verses)

Changes:
- Updated process_v1_data_fixed.py to normalize line breaks:
  - Single verses: Replace \n\n with \n (no gaps within verse)
  - Verse ranges: Normalize each verse, then join with \n\n (proper separation)
- Regenerated all v4 common and author JSON files
- Updated both web and Flutter app data

Result:
- Single verses now display with proper line spacing
- Verse ranges (e.g., 4-6) now have clear visual separation between each verse
Fixed bug in v1 source data where verse 5 had 3 lines instead of 2:
- Verse 5 had verse 6's first line incorrectly appended
- Verse 6 was missing its first line

Now all verses correctly have 2 lines each matching Sanskrit structure:
- Verse 4: 2 lines ✅
- Verse 5: 2 lines ✅ (was 3 lines ❌)
- Verse 6: 2 lines ✅ (was 1 line ❌)

Changes:
- Fixed v1/verse.json source data
- Regenerated all v4 data files
- Updated both web and Flutter app data
Fixed data corruption in v1 source where transliteration lines were
misattributed between verses:
- Verse 1.20: Removed extra line that belonged to verse 1.21
- Verse 1.21: Added missing middle line

This ensures proper alignment between Sanskrit and transliteration,
with verse 1.21-22 now correctly showing 5 transliteration lines
(speaker designation + 2 lines from v21 + 2 lines from v22).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant